This invention relates to an image processing system capable of providing, from a plurality of input images including images each focused at a corresponding level of a thick object (sample) with different thicknesses, an image appearing as if it is focused at each level of the object, i.e. as if it has an expanded focal depth.
To analyze an object using its input image, the image must have a high brightness, a high resolution and an expanded focal depth. In general, an image forming optical system which employs an optical element with a large aperture is necessary to optically pick up an image with a high resolution, a high magnification and a high brightness.
In an image forming optical element represented by a lens, however, the greater the aperture, the shallower the focal depth. Accordingly, if an image forming optical element with a great aperture is used for securing a high magnification, only the portion of an object which is situated in a position along the optical axis is in focus, and the other portions of the object which are situated in the other positions along the optical axis are out of focus. This is because the focal depth is shallow.
To avoid the above, in the field of optical devices such as a microscope, camera, an endoscope, etc., various means are conceived for increasing the focal depth. Japanese Patent Application KOKAI Publication No. 1-309478, for example, discloses a technique for adding a plurality of images respectively focused in different positions along the optical axis, and subjecting the added images to restoring processing using a restoration filter to thereby restore a single image focused in different positions along the optical axis. Moreover, Japanese Patent Application KOKAI Publications Nos. 1-309478 and 2-192276 disclose a technique using no restoration filter for subjecting a plurality of images to Fourier transform, then to summation, and to reverse Fourier transform, thereby restoring a single image focused in different positions along the optical axis.
At the time of using the above-described focal depth expanding means or restoration filter to restore a microscopic image, it is necessary to consider various matters peculiar to a microscope. Specifically, in the case of observing a sample with a microscope, the observer first searches, at a low magnification, a to-be-observed point within a wide range of the sample, and thereafter observes the point in detail at a high magnification by mainly switching objective lenses from one to another. The observer repeats this operation. In other words, at the time of using a microscope, its magnification is repeatedly switched, usually.
The image forming state of an image pick-up element incorporated in a microscope as above will now be described. The numerical aperture (NA') of the emission side of an image forming optical system depends upon the degree of magnification. Supposing that"integrated point image intensity distribution" means integration of all the point image intensity distributions of a certain point assumed from the state where the point is in true focus to the state where it is infinitely out of focus, the range of the integrated point image intensity distribution on the image pick-up element varies in accordance with the degree of magnification.
The restoration filter used for the focal depth expansion is determined on the basis of the range of the integrated point image intensity distribution on the image pick-up element. This is because the restoration filter provides an excellent restoration image only when the range of the integrated point image intensity distribution matches the range of the design value distribution of the restoration filter.
Referring to FIG. 8, the above matter will be explained specifically.
In FIG. 8, the abscissa indicates standardized distances from the center of a point image, while the ordinate indicates the intensities of the point image with a center intensity set to"1". The broken line indicates an ideal point image intensity distribution in a focal point, and each solid line indicates a restored point image intensity distribution obtained when the restoration filter is designed to meet with an integrated point image intensity distribution range which is 1/.beta. of the range of the actual integrated point image intensity distribution.
As is understood from FIG. 8, an ideal restoration image can be obtained when .beta.=1. However, when .beta.&lt;1, an image with a ringing edge portion is obtained, while when .beta.&gt;1, an image is obtained which has an intensity distribution range larger than an ideal point image intensity distribution range.
As explained above, the restoration filter provides an excellent restoration image only when the integrated point image intensity distribution range coincides with the design value distribution range of the restoration filter. Accordingly, a single restoration filter cannot provide excellent restoration images at different magnification degrees of a microscope.
In addition, the range of the integrated point image intensity distribution on the image pick-up element is determined not only by the numerical aperture NA' of the emission side of the image forming optical system, but also by the wavelength used in the system and the pupil function thereof. Thus, also when these factors do not coincide with the design values of the restoration filter, no excellent restoration image can be obtained. The use of a single restoration filter in a color CCD, etc. employed as the image pick-up element will cause a blurred restoration image.
Furthermore, the following is included in the problems peculiar to the microscope:
At the time of picking up an image of a sample using a microscope, the sample is often observed at a high magnification in order to examine a very small portion thereof. In this case, the range of the integrated point image intensity distribution on the image pick-up element is inevitably large. The restoration filter is usually set to a matrix size of about seven rows and seven columns in light of cost and time required to calculation. If the range of the integrated point image intensity distribution exceeds the size of the matrix, no excellent restoration image is obtainable.
If, on the other hand, the matrix size of the restoration filter is set larger than the range of the integrated point image intensity distribution in order to secure an excellent restoration image, the time and memory capacity required to calculation increase in proportion to the square value of the matrix size, which is significantly disadvantageous.